Quantile Random Forest

Most regression models give you a single number: the expected value of the target given the inputs. That’s often what you want, but it throws away information. Quantile Random Forest (QRF) is an extension of the standard Random Forest that estimates the entire conditional distribution of the target — not just its mean, but any quantile you ask for.

How QRF differs from standard Random Forest

In a regular Random Forest, each tree makes a prediction, and the final output is the average across trees. QRF keeps more of the raw information. During training, the trees are built the same way — bootstrap samples, random feature subsets at each split — but instead of storing only the mean response in each leaf, the tree retains all training observations that fall in that leaf.

At prediction time, you pass a new point through all trees, collect the training observations from each matching leaf, and pool them into a single set of values. That set is treated as a sample from the conditional distribution of the target. From there, any quantile is just a percentile of that collection: the 0.1 quantile is the 10th percentile, the 0.9 quantile is the 90th percentile, and so on.

This means QRF doesn’t require you to specify which quantiles you care about at training time. You train once and query the distribution at any point you want afterward.

Why this is useful

The main reason to use QRF over standard Random Forest is when the variance of the target changes across the input space — what statisticians call heteroscedasticity. If uncertainty is higher in some regions of the feature space than others, the mean alone doesn’t capture that. QRF gives you the spread as well.

More concretely: instead of predicting “this customer’s lifetime value is €240”, you can say “the median is €240, but the 90th percentile is €700 — this is a high-variance prediction.” That distinction matters if you’re making decisions based on the output.

Prediction intervals are a direct application. By taking, say, the 5th and 95th percentile predictions, you get a range that should contain the true value 90% of the time (conditional on the inputs). Unlike parametric prediction intervals, QRF makes no assumption about the shape of the residual distribution — it estimates the interval directly from the data.

Robustness to outliers is another benefit. Because quantiles are inherently resistant to extreme values, QRF estimates of the median or other quantiles are more stable than the mean when the target has a heavy tail.

Applications

Financial forecasting: predicting a distribution over returns rather than a point estimate is directly useful for risk management. Value at Risk, for example, is just a quantile of the loss distribution.

Healthcare: modeling the distribution of patient outcomes allows clinicians to identify not just the expected response to a treatment, but also the tail risk — the probability of an unusually bad outcome.

Environmental modeling: concentrations of pollutants, rainfall amounts, or temperatures all have distributions that matter for risk assessment. A 95th percentile estimate of a pollutant level is often the relevant quantity for regulatory purposes, not the mean.

Operations and demand planning: knowing the 90th percentile of demand, not just the average, is essential for setting safety stock levels. Optimizing for the mean will leave you understocked half the time.

QRF is available in R via the quantregForest package and in Python through quantile-forest and scikit-garden. The interface is familiar if you’ve used Random Forests before — the main difference is that the predict call takes a quantiles argument rather than returning a single value.